Add release notes agentic workflow by richlander · Pull Request #10334 · dotnet/core

richlander · 2026-04-01T03:41:37Z

Summary

Adds an agentic workflow that automatically generates and maintains .NET release notes by analyzing the VMR (dotnet/dotnet) source-manifest diffs. Changes.json (complete list of commits and PRs) is another generated artifact that can be used in other works. Is largely schema-compatible with cve.json.

Architecture (3 layers)

dotnet-release tool (deterministic) — diffs src/source-manifest.json between release refs to produce changes.json, a comprehensive manifest of every PR that shipped
AI agent (editorial) — reads changes.json and writes curated markdown release notes, guided by reference docs in .github/skills/release-notes/references/
Agentic workflow (orchestration) — runs nightly, manages branch lifecycle, respects human edits, responds to PR comments

Key capabilities

Multi-milestone — discovers all active milestones (e.g., Preview 4 and Preview 5 simultaneously) and creates a branch + PR for each
Nightly re-validation — base/head refs are re-checked every run as VMR tags appear, release branches are created, and main bumps
Human-in-the-loop — respects edits on branches, reads and responds to PR comments, asks when unsure
Deterministic milestone detection — 3-signal algorithm using releases.json, VMR Versions.props, and VMR tags

Files

Path	Purpose
`.github/workflows/release-notes.md`	Agentic workflow definition
`.github/workflows/release-notes.lock.yml`	Compiled workflow (generated)
`.github/skills/release-notes/DESIGN.md`	System design document
`.github/skills/release-notes/SKILL.md`	Skill metadata
`.github/skills/release-notes/references/*.md`	Reference docs (schema, templates, editorial rules, etc.)

Testing

Tested dotnet-release generate changes (P1→P2: 1,389 changes; P2→P3: 2,259 changes)
Tested milestone detection algorithm against live VMR state
Full E2E test: generated all component markdown files from changes.json
Compiled workflow with gh aw compile — passes validation

Dependencies

dotnet-release v2.0.0+ (the generate changes command)
GitHub Agentic Workflows runtime

Cron-driven GitHub Agentic Workflow that maintains release notes branches for .NET preview/RC/GA releases. Architecture: - dotnet-release generate changes tool produces changes.json (deterministic, from VMR source-manifest.json diffs) - Agent reads changes.json and writes curated markdown for high-value features - One PR per release milestone, maintained incrementally - Respects human edits on the branch Skill references (6 docs, ~700 lines total): - quality-bar.md: what good release notes look like - vmr-structure.md: VMR branches, tags, source-manifest.json - changes-schema.md: changes.json schema reference - component-mapping.md: components, product slugs, output files - format-template.md: markdown document structure - editorial-rules.md: tone, attribution, naming Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Describes the three-layer architecture (tool → agent → workflow), the source-manifest.json mechanism, branch lifecycle, and design rationale. Links from SKILL.md for discoverability. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Tested dotnet-release v1.1.0 generate changes against the VMR (P1→P2: 1,389 changes across 21 repos). Updated docs to reflect the actual tool output: - Remove aspirational product/package fields from changes-schema.md (tool outputs repo, not product slugs) - Add labels field (optional, via --labels flag) - Fix workflow CLI syntax (positional path, --base/--head) - Update component-mapping.md: agent does repo→file routing - Remove resolved open question about package mapping - Update example JSON and jq queries to match real output Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

vstest is the only microsoft org repo in the VMR manifest and is already on the infra skip list. SAML token requirements for the microsoft org can be addressed later. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Tested --labels flag: 776/1389 changes got labels (56% coverage). Best coverage: runtime (94%), roslyn (96%), winforms/wpf (100%). Lower coverage on infra-heavy repos as expected. Corrected the schema docs — the product field IS output by the tool (repo-level mapping), just empty for infra repos. Added product back to the schema with accurate description. Added labels field documentation and updated examples/queries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace heuristic branch-scanning with a deterministic algorithm using three data sources: 1. releases.json → what has shipped 2. VMR eng/Versions.props on main → current development iteration 3. VMR tags → base refs for generating diffs Decision: if Versions.props iteration > latest shipped → that's the target. Exact commands provided in workflow steps 1-2. Updated DESIGN.md (milestone detection section), vmr-structure.md (version metadata + base tag discovery), and the workflow file (deterministic steps 1 and 2 with concrete examples). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Once main is bumped to the next iteration, it contains preview N+1 work. The agent must check if a VMR tag exists for the target milestone: - No tag → in development, use --head main - Tag exists → finalized, use --head <tag> Updated workflow step 1e and step 2 with both code paths. Added Head ref selection section to DESIGN.md explaining why this matters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The general 'What'\''s new in .NET' link belongs in README.md, not in every component file. Component files should link to their specific docs page when one exists (runtime, libraries, SDK, ASP.NET Core, C#, EF Core). Added the dotnet/docs overview source URL so the agent can discover component-specific links. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

v1.2.0 now maps every source-repo change to its VMR codeflow commit. The new dotnet_commit field on each change entry keys into the commits dict for the dotnet/dotnet commit. Multiple source PRs typically share the same VMR commit (codeflow batches). Updated schema docs, example JSON, and field descriptions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- commit field is now VMR commit (was source repo) - dotnet_commit removed (replaced by commit) - local_repo_commit added for source repo commit - product field absent for infra repos (not empty string) - Updated example and jq queries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Multi-milestone: discover all active milestones (P4 and P5 simultaneously) - Re-validate base/head refs every nightly run (tags appear, branches create, main bumps) - Branch-per-milestone on dotnet/core (release-notes/11.0-preview4, etc.) - Human interaction: respect edits, respond to PR comments, ask when unsure - Handle transitions: main bumps, new tags, release branches, PR merges - Increased safe-output limits for multi-milestone support - Updated DESIGN.md with multi-milestone discovery, human interaction model Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Fix invalid toolsets: contents→repos, code_search→search - Add compiled release-notes.lock.yml - Add .github/aw/actions-lock.json (gh aw artifact) - Add linguist-generated gitattributes for lock files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The inline example in this file is the authoritative reference. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Six examples from .NET 9/10 preview release notes demonstrating three styles: - Short: exception handler diagnostics, container insecure registries - Medium: extension operators (C#), PipeReader JSON support - Long: post-quantum cryptography, OpenIdConnect PAR Added to reference docs list in workflow and DESIGN.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Three styles of JIT feature descriptions: - Array Enumeration De-Abstraction (progressive benchmarks, optimization narrative) - Struct Arguments codegen (before/after assembly comparison) - Loop Optimizations (metric-heavy prose, no code) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The agent should recognize when a feature description needs content it can't generate (benchmark data, definitive code samples, domain expertise) and post PR comments requesting it from feature authors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace monolithic examples.md with per-component files: - examples/README.md — principles index, component directory - examples/runtime.md — 3 JIT examples with inlined content - examples/aspnetcore.md — 3 examples (short/medium/long) - examples/csharp.md — extension operators - examples/sdk.md — container insecure registries - examples/libraries.md — post-quantum cryptography Each file includes the actual example text and links back to the original source file. Agent loads only the relevant component file when writing, saving context tokens. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

New format per file: ## Heading (H2) Actual content (copy/paste from original) --- Source: link to original Commentary: why it was selected Why it works: what makes it effective --- Clear separation between the example content and the editorial commentary. Content reads exactly as it would in release notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- id field is now string (repo@commit format) — globally unique - Updated schema docs and example JSON - Removed trailing blank lines in example metadata blocks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

More frequent runs mean faster feedback loops on PR comments and quicker pickup of new VMR content. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replaces SuppressDiagnosticsCallback (prose-only) with StatusCodeSelector (one paragraph + clean code sample). Better exemplar for 'short + code'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The 'guest introduction' style restructures the note around attribution instead of the feature. Prefer: explain the feature, then credit at end. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

'Snippet' in a programming context suggests code. Say what you mean plainly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ched

jeffhandley

@richlander I merged main and went ahead and applied all of my feedback with a series of commits. Feel free to unwind them and force push if you'd like but I figured pushing the commits was a more efficient way of submitting all of my feedback.

I'm running the workflow on my fork now (run) and I'd like to review the results that are generated before hitting approve, but if the results match what we're looking for the implementation looks all right.

The aspects I'm particularly going to be looking for in the produced results are:

How does it do for libraries without directly referencing the API Diff? Presumably dotnet-inspect is going to substitute that but the api diff reports are a dense diff format that seemed to be ideal for getting the API changes into context.
Taking the dependency on external tools/repos for gathering the changes. Ideally that would all be self-contained, IMHO; but I understand those tools are still evolving and this is a good test bed for them as well.
Not having component-specific references for teams to provide component-specific guidance into the data gathering and authoring. I understand what you're after here though and we can add that back in if needed.

There are some places where the workflow, skill, and the design doc repeat statements/content/logic. In the commits I pushed, I dialed some of that back a little but there is still repeated statements across those pieces of content that will be prone to drift over time. For example, the schedule info was repeated across several places and it was inconsistent. I updated those references for consistency and to reduce the "sources of truth" of the info--including updating the cron schedule to be daily ~ 9am PDT instead of every 6 hours.

Also with my commits, I integrated the select-copilot-pat logic that allows us to have a pool of PATs to draw from instead of relying on one person's PAT. I did the initial setup of that so the workflow will run. I removed the Open Question about the PAT/scope that you had as this addresses that topic.

jeffhandley · 2026-04-03T07:35:17Z

.github/skills/release-notes/DESIGN.md

+3. **Queries GitHub compare API** for each changed component to enumerate merged PRs in the commit range
+4. **Maps repos to product slugs** using a built-in taxonomy (e.g., `runtime` → `dotnet-runtime`; infra repos get no product)
+5. **Fetches PR labels** when `--labels` is provided (useful for agent categorization)
+6. **Cross-references CVEs** when `--cve-repo` is provided (loads `cve.json` from the `release-index` branch)


I'm curious how you intend to reference CVE info giving the timing of the CVE publications, security fixes, and the authoring of release notes. I didn't expect to see any security- or CVE-related topics in this skill/workflow.

This doesn't apply to previews. For servicing releases, this system will likely be run behind the firewall.

jeffhandley · 2026-04-03T08:33:31Z

Edge case testing, but on my fork I deleted the preview-2 release notes so I could regenerate them with this implementation and compare the output to what we publsihed. The workflow ended up failing.

https://github.com/jeffhandley/dotnet-core/actions/runs/23938625467/job/69820753316#step:9:115

Error: ✗ Message 1 (push_to_pull_request_branch) failed: push-to-pull-request-branch with target "triggering" requires pull request context
Warning: ⚠️ Code push operation 'push_to_pull_request_branch' failed — remaining safe outputs will be cancelled

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jeffhandley

Still reading but sharing some comments while I make progress.

jeffhandley · 2026-04-06T22:16:28Z

.github/skills/api-diff/SKILL.md

 ```

-Set an initial wait of at least 300 seconds — the script takes several minutes. Use `read_powershell` to poll for completion. The script does not print a final "done" message; it exits after generating a README.md in the output folder. After completion, summarize the results: how many diff files were generated and where.
+Set an initial wait of at least 300 seconds — the script takes several minutes. After completion, summarize the results: how many diff files were generated and where.


Curious why you're removing this sentence. I had added it after the agent kept tripping up on how to consume the output from the tool, and this made it more reliable.

That seems like overfit. Did you find that the agent needs that information?

jeffhandley · 2026-04-06T22:29:38Z

.github/skills/release-notes/STATUS.md

I'm not sure if this file needs to be committed or if we should just include this as a comment on the PR.

.github/skills/release-notes/references/editorial-rules.md

.github/skills/release-notes/references/examples/README.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jeffhandley

Just a few additional comments; approving for when those are addressed. Thanks for all the effort on this!

.github/skills/release-notes/references/feature-scoring.md

.github/workflows/release-notes.md

Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Update the workflow docs, repo skills, and locked workflow permissions to use the renamed maintainer CLI and package name while keeping dotnet-release for the public graph-query tool. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

richlander and others added 30 commits March 26, 2026 16:22

Remove stale gist link from changes schema docs

4d7add2

The inline example in this file is the authoritative reference. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Run every 6 hours instead of daily

9c0d47a

More frequent runs mean faster feedback loops on PR comments and quicker pickup of new VMR content. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix commentary: callback example would benefit from code sample

ef90d9b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace short ASP.NET example: StatusCodeSelector with code snippet

5016b21

Replaces SuppressDiagnosticsCallback (prose-only) with StatusCodeSelector (one paragraph + clean code sample). Better exemplar for 'short + code'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add active voice principle to examples guide

19c1636

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add style note: simplify awkward phrasing in PipeReader example

1d92154

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Mark PAR example as anti-pattern; add Title/Content/Credit principle

05ae36a

The 'guest introduction' style restructures the note around attribution instead of the feature. Prefer: explain the feature, then credit at end. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add style note: simplify transition to rule bullets in C# example

60b1c8a

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add style note: active voice for MLDsa ease-of-use intro

50518fe

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add principle: link PRs/issues in org/repo #number format

f1c1a7e

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add principle: avoid jargon and ambiguous words

48940cc

'Snippet' in a programming context suggests code. Say what you mean plainly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add style note: don't bury the lede with backward-looking passive

63e49d2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jeffhandley added 7 commits April 2, 2026 23:54

Initialize Agentic Workflows agent

7a80b7d

Set up the select-copilot-pat integration into agentic workflows

9d264c1

Integrate select-copilot-pat into release-notes workflow

bda5c7a

Move labeler workflow readme to avoid it being recognized as a workflow

482744f

Apply review feedback. Set schedule to be daily around 9am PDT.

43e2e1b

Do not run the release-notes workflow on forks unless manually dispat…

7ddb31b

…ched

Set release-notes workflow timeout to 2 hours

1fb31f1

jeffhandley reviewed Apr 3, 2026

View reviewed changes

jeffhandley and others added 5 commits April 3, 2026 17:06

Merge branch 'main' into release-notes-agentic-workflow

ca7d28a

Split release notes skill pipeline

d100e65

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Refine release notes scoring guidance

a7a8711

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Clarify release-notes editorial guidance

6c2dd1c

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Prefer stronger verbs in release-note headings

a00e63f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

richlander requested review from leecow and rbhanda as code owners April 6, 2026 21:26

jeffhandley reviewed Apr 6, 2026

View reviewed changes

Refine release-notes workflow guidance

9242e31

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

richlander force-pushed the release-notes-agentic-workflow branch from 12a40b4 to 9242e31 Compare April 6, 2026 22:42

Remove redundant release-notes docs

ecacff1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

jeffhandley approved these changes Apr 6, 2026

View reviewed changes

richlander and others added 5 commits April 6, 2026 16:21

Update .github/workflows/release-notes.md

f56f9b5

Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>

Update .github/workflows/release-notes.md

0b545af

Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>

Deduplicate editorial rubric guidance

9faac3b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Clarify preview and GA api-diff labels

d9655ed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

richlander merged commit 3dfeeca into main Apr 6, 2026
6 of 8 checks passed

richlander deleted the release-notes-agentic-workflow branch April 6, 2026 23:58

jeffhandley mentioned this pull request Apr 13, 2026

Refactor libraries-release-notes into shared release-notes skill #10282

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add release notes agentic workflow#10334

Add release notes agentic workflow#10334
richlander merged 67 commits intomainfrom
release-notes-agentic-workflow

richlander commented Apr 1, 2026 •

edited

Loading

Uh oh!

jeffhandley left a comment •

edited

Loading

Uh oh!

jeffhandley Apr 3, 2026

Uh oh!

richlander Apr 6, 2026

Uh oh!

jeffhandley commented Apr 3, 2026

Uh oh!

jeffhandley left a comment

Uh oh!

jeffhandley Apr 6, 2026

Uh oh!

richlander Apr 6, 2026

Uh oh!

jeffhandley Apr 6, 2026

Uh oh!

Uh oh!

Uh oh!

jeffhandley left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

richlander commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Architecture (3 layers)

Key capabilities

Files

Testing

Dependencies

Uh oh!

jeffhandley left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffhandley Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

richlander Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

jeffhandley commented Apr 3, 2026

Uh oh!

jeffhandley left a comment

Choose a reason for hiding this comment

Uh oh!

jeffhandley Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

richlander Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

jeffhandley Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jeffhandley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

richlander commented Apr 1, 2026 •

edited

Loading

jeffhandley left a comment •

edited

Loading